No model represents the true process that generated the outcomes.
Information is the “distance” between a prospective models and the true model
\[D_{KL}(M_{true} | M_1) \neq D_{KL}(M_1 | M_{true})\]
KL information doesn’t aid directly in model evaluation (what is the true model?)
\[E = \log[p_{M_1}(y)]\]
elpd is estimated by information criteria or leave-one-out cross-validation (next lecture)
options(brms.backend = "cmdstanr",
mc.cores = 4)
fm1 <- brm(Energy ~ 1, data = M,
prior = prior(normal(0, 3), class = Intercept), iter = 2e4, refresh = 0)
fm2 <- brm(Energy ~ Caste, data = M,
prior = prior(normal(0, 3), class = b), iter = 2e4, refresh = 0)
fm3 <- brm(Energy ~ Mass, data = M,
prior = prior(normal(0, 3), class = b), iter = 2e4, refresh = 0)
fm4 <- brm(Energy ~ Mass + Caste, data = M,
prior = prior(normal(0, 3), class = b), iter = 2e4, refresh = 0)
fm5 <- brm(Energy ~ Mass * Caste, data = M,
prior = prior(normal(0, 3), class = b), iter = 2e4, refresh = 0)
Computed from 40000 by 35 log-likelihood matrix
Estimate SE
elpd_waic -16.7 3.8
p_waic 1.7 0.5
waic 33.4 7.7
Computed from 40000 by 35 log-likelihood matrix
Estimate SE
elpd_waic -17.6 4.2
p_waic 2.7 0.7
waic 35.2 8.4
2 (5.7%) p_waic estimates greater than 0.4. We recommend trying loo instead.
| Weight | |
|---|---|
| fm1 | 0.0005 |
| fm2 | 0.0002 |
| fm3 | 0.0387 |
| fm4 | 0.4942 |
| fm5 | 0.4664 |